Add proxy tool calls option to chat models #1398

tzolov · 2024-09-23T14:15:15Z

This commit introduces a new proxyToolCalls option for various chat
models in the Spring AI project. When enabled, it allows the client to
handle function calls externally instead of being processed internally
by Spring AI.

The change affects multiple chat model implementations, including:
AnthropicChatModel
AzureOpenAiChatModel
MiniMaxChatModel
MistralAiChatModel
MoonshotChatModel
OllamaChatModel
OpenAiChatModel
VertexAiGeminiChatModel
ZhiPuAiChatModel

The proxyToolCalls option is added to the respective chat options
classes and integrated into the AbstractToolCallSupport class for
consistent handling across different implementations.

The proxyToolCalls option can be set either programmatically via
the ChatOptions.builder().withProxyToolCalls() method
or the spring.ai..chat.options.proxy-tool-calls
application property.

Documentation for the new option is also updated in the relevant
Antora pages.

Resolves #1367

This commit introduces a new proxyToolCalls option for various chat models in the Spring AI project. When enabled, it allows the client to handle function calls externally instead of being processed internally by Spring AI. The change affects multiple chat model implementations, including: AnthropicChatModel AzureOpenAiChatModel MiniMaxChatModel MistralAiChatModel MoonshotChatModel OllamaChatModel OpenAiChatModel VertexAiGeminiChatModel ZhiPuAiChatModel The proxyToolCalls option is added to the respective chat options classes and integrated into the AbstractToolCallSupport class for consistent handling across different implementations. The proxyToolCalls option can be set either programmatically via the <ModelName>ChatOptions.builder().withProxyToolCalls() method or the spring.ai.<model-name>.chat.options.proxy-tool-calls application property. Documentation for the new option is also updated in the relevant Antora pages. Resolves spring-projects#1367

Add ITs for proxy tool calls

markpollack · 2024-09-23T20:07:13Z

spring-ai-core/src/main/java/org/springframework/ai/chat/client/ChatClient.java

@@ -74,9 +74,9 @@ static Builder builder(ChatModel chatModel, ObservationRegistry observationRegis

 	ChatClientRequestSpec prompt();

-	ChatClientPromptRequestSpec prompt(String content);
+	ChatClientRequestSpec prompt(String content);


I didn't think that allowing the 'sealed' prompts to participate in the rest of the fluent api would be this straightforward, nice!

I just added docs today wrt to the overloads being 'sealed' so I'll update that and seems like we can delete the interface ChatClientPromptRequestSpec

markpollack · 2024-09-23T20:19:20Z

...enai/src/test/java/org/springframework/ai/openai/chat/proxy/OllamaWithOpenAiChatModelIT.java

Likely unrelated to the changes, but I got a few failing tests. Adding stack traces here for future reference. As this is manual smoke test, I'll merge as it won't break the build.

2024-09-23T16:08:15.951-04:00 ERROR 18871 --- [onPool-worker-2] o.s.ai.chat.model.MessageAggregator : Aggregation Error org.springframework.web.reactive.function.client.WebClientResponseException$BadRequest: 400 Bad Request from POST http://localhost:32769/v1/chat/completions at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:321) ~[spring-webflux-6.1.12.jar:6.1.12] Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s): *__checkpoint ⇢ 400 BAD_REQUEST from POST http://localhost:32769/v1/chat/completions [DefaultWebClient] Original Stack Trace: at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:321) ~[spring-webflux-6.1.12.jar:6.1.12] at org.springframework.web.reactive.function.client.DefaultClientResponse.lambda$createException$1(DefaultClientResponse.java:214) ~[spring-webflux-6.1.12.jar:6.1.12] at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxOnErrorReturn$ReturnSubscriber.onNext(FluxOnErrorReturn.java:162) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:299) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:337) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPublish$PublishSubscriber.checkTerminated(FluxPublish.java:634) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPublish$PublishSubscriber.drain(FluxPublish.java:494) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPublish$PublishSubscriber.onComplete(FluxPublish.java:355) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onComplete(FluxContextWrite.java:126) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onComplete(FluxMapFuseable.java:350) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:371) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onComplete(FluxFlattenIterable.java:273) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.adapter.JdkFlowAdapter$SubscriberToRS.onComplete(JdkFlowAdapter.java:160) ~[reactor-core-3.6.9.jar:3.6.9] at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.complete(ResponseSubscribers.java:933) ~[java.net.http:na] at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.lambda$new$1(ResponseSubscribers.java:864) ~[java.net.http:na] at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[na:na] at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.lambda$subscribe$3(ResponseSubscribers.java:961) ~[java.net.http:na] at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:757) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:735) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2182) ~[na:na] at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.subscribe(ResponseSubscribers.java:957) ~[java.net.http:na] at reactor.adapter.JdkFlowAdapter$FlowPublisherAsFlux.subscribe(JdkFlowAdapter.java:68) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPublish.connect(FluxPublish.java:106) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxAutoConnect.subscribe(FluxAutoConnect.java:62) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.9.jar:3.6.9] at org.springframework.http.client.reactive.AbstractClientHttpResponse$SingleSubscriberPublisher.subscribe(AbstractClientHttpResponse.java:112) ~[spring-web-6.1.12.jar:6.1.12] at reactor.core.publisher.FluxSource.subscribe(FluxSource.java:71) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.Flux.subscribe(Flux.java:8848) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:196) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onNext(FluxPeekFuseable.java:854) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.complete(MonoIgnoreThen.java:294) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onNext(MonoIgnoreThen.java:188) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoCompletionStage$MonoCompletionStageSubscription.apply(MonoCompletionStage.java:121) ~[reactor-core-3.6.9.jar:3.6.9] at reactor.core.publisher.MonoCompletionStage$MonoCompletionStageSubscription.apply(MonoCompletionStage.java:67) ~[reactor-core-3.6.9.jar:3.6.9] at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[na:na] at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:614) ~[na:na] at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:844) ~[na:na] at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) ~[na:na] at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[na:na] at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[na:na] at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[na:na] at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[na:na] at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[na:na] org.springframework.web.reactive.function.client.WebClientResponseException$BadRequest: 400 Bad Request from POST http://localhost:32769/v1/chat/completions at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:321) Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below: Error has been observed at the following site(s): *__checkpoint ⇢ 400 BAD_REQUEST from POST http://localhost:32769/v1/chat/completions [DefaultWebClient] Original Stack Trace: at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:321) at org.springframework.web.reactive.function.client.DefaultClientResponse.lambda$createException$1(DefaultClientResponse.java:214) at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106) at reactor.core.publisher.FluxOnErrorReturn$ReturnSubscriber.onNext(FluxOnErrorReturn.java:162) at reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122) at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:299) at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:337) at reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097) at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145) at reactor.core.publisher.FluxPublish$PublishSubscriber.checkTerminated(FluxPublish.java:634) at reactor.core.publisher.FluxPublish$PublishSubscriber.drain(FluxPublish.java:494) at reactor.core.publisher.FluxPublish$PublishSubscriber.onComplete(FluxPublish.java:355) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onComplete(FluxContextWrite.java:126) at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onComplete(FluxMapFuseable.java:350) at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:371) at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724) at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onComplete(FluxFlattenIterable.java:273) at reactor.adapter.JdkFlowAdapter$SubscriberToRS.onComplete(JdkFlowAdapter.java:160) at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.complete(ResponseSubscribers.java:933) at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.lambda$new$1(ResponseSubscribers.java:864) at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.lambda$subscribe$3(ResponseSubscribers.java:961) at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:757) at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:735) at java.base/java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2182) at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.subscribe(ResponseSubscribers.java:957) at reactor.adapter.JdkFlowAdapter$FlowPublisherAsFlux.subscribe(JdkFlowAdapter.java:68) at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:68) at reactor.core.publisher.FluxPublish.connect(FluxPublish.java:106) at reactor.core.publisher.FluxAutoConnect.subscribe(FluxAutoConnect.java:62) at reactor.core.publisher.Flux.subscribe(Flux.java:8848) at org.springframework.http.client.reactive.AbstractClientHttpResponse$SingleSubscriberPublisher.subscribe(AbstractClientHttpResponse.java:112) at reactor.core.publisher.FluxSource.subscribe(FluxSource.java:71) at reactor.core.publisher.Flux.subscribe(Flux.java:8848) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:196) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180) at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onNext(FluxPeekFuseable.java:854) at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.complete(MonoIgnoreThen.java:294) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onNext(MonoIgnoreThen.java:188) at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) at reactor.core.publisher.MonoCompletionStage$MonoCompletionStageSubscription.apply(MonoCompletionStage.java:121) at reactor.core.publisher.MonoCompletionStage$MonoCompletionStageSubscription.apply(MonoCompletionStage.java:67) at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:614) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:844) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Suppressed: java.lang.Exception: #block terminated with an error at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:104) at reactor.core.publisher.Mono.block(Mono.java:1779) at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.streamingMultiModalityImageUrl(OllamaWithOpenAiChatModelIT.java:365) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

and

2024-09-23T16:08:21.725-04:00 WARN 18871 --- [ main] o.s.ai.openai.OpenAiChatModel : Removing streamOptions from the request as it is not a streaming request! java.lang.AssertionError: Expecting actual: 0L to be greater than: 0L at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.streamingWithTokenUsage(OllamaWithOpenAiChatModelIT.java:144) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

and

java.lang.IllegalStateException: No function callback found for function name: getCurrentWeathers at org.springframework.ai.chat.model.AbstractToolCallSupport.executeFunctions(AbstractToolCallSupport.java:197) at org.springframework.ai.chat.model.AbstractToolCallSupport.handleToolCalls(AbstractToolCallSupport.java:139) at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:267) at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.functionCallTest(OllamaWithOpenAiChatModelIT.java:279) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

and

2024-09-23T16:16:08.571-04:00 INFO 18871 --- [ main] o.s.ai.openai.chat.OpenAiChatModelIT : Response: To get the current weather for multiple locations, I would iterate through an array of locations with their respective latitude, longitude, and return the current temperature for each location. Here's a Python sample code using your function named getCurrentWeather: ```python locations = [ {"location": "San Francisco, CA", "lat": 37.7749, "lon": -122.4194}, {"location": "Tokyo, JP", "lat": 35.6895, "lon": 139.6917}, {"location": "Paris, FR", "lat": 48.8586, "lon": 2.3508} ] unit = "C" for location in locations: result = getCurrentWeather(location["lat"], location["lon"], unit="C") print("{}\t{}°C".format(*result))

With this code, you will get the current temperature in each city you provided. To execute the script, replace the getCurrentWeather function with your actual implementation.

This code would output something like:

San Francisco, CA 14°C Tokyo 30°C Paris 16°C

java.lang.AssertionError:
Expecting actual:
" To get the current weather for multiple locations, I would iterate through an array of locations with their respective latitude, longitude, and return the current temperature for each location. Here's a Python sample code using your function named getCurrentWeather:

locations = [ {"location": "San Francisco, CA", "lat": 37.7749, "lon": -122.4194}, {"location": "Tokyo, JP", "lat": 35.6895, "lon": 139.6917}, {"location": "Paris, FR", "lat": 48.8586, "lon": 2.3508} ] unit = "C" for location in locations: result = getCurrentWeather(location["lat"], location["lon"], unit="C") print("{}\t{}°C".format(*result))

With this code, you will get the current temperature in each city you provided. To execute the script, replace the getCurrentWeather function with your actual implementation.

This code would output something like:

San Francisco, CA 14°C Tokyo 30°C Paris 16°C ```" to contain: ["30", "10", "15"] but could not find: ["10", "15"] at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.streamFunctionCallTest(OllamaWithOpenAiChatModelIT.java:315) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

and

2024-09-23T16:16:08.590-04:00 WARN 18871 --- [ main] o.springframework.ai.retry.RetryUtils : Retry error. Retry count:1 org.springframework.ai.retry.NonTransientAiException: 400 - {"error":{"message":"invalid image input","type":"invalid_request_error","param":null,"code":null}} at org.springframework.ai.retry.RetryUtils$2.handleError(RetryUtils.java:77) ~[classes/:na] at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.StatusHandler.lambda$fromErrorHandler$1(StatusHandler.java:71) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.StatusHandler.handle(StatusHandler.java:146) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.applyStatusHandlers(DefaultRestClient.java:696) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:200) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:683) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntityInternal(DefaultRestClient.java:653) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntity(DefaultRestClient.java:642) ~[spring-web-6.1.12.jar:6.1.12] at org.springframework.ai.openai.api.OpenAiApi.chatCompletionEntity(OpenAiApi.java:1022) ~[classes/:na] at org.springframework.ai.openai.OpenAiChatModel.lambda$call$1(OpenAiChatModel.java:226) ~[classes/:na] at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:344) ~[spring-retry-2.0.8.jar:na] at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:217) ~[spring-retry-2.0.8.jar:na] at org.springframework.ai.openai.OpenAiChatModel.lambda$call$3(OpenAiChatModel.java:226) ~[classes/:na] at io.micrometer.observation.Observation.observe(Observation.java:565) ~[micrometer-observation-1.13.3.jar:1.13.3] at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:223) ~[classes/:na] at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.multiModalityImageUrl(OllamaWithOpenAiChatModelIT.java:345) ~[test-classes/:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[na:na] at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) ~[junit-platform-commons-1.10.3.jar:1.10.3] at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:226) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:204) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:142) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:110) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992) ~[na:na] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na] at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[na:na] at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na] at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[na:na] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na] at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[na:na] at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) ~[na:na] at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[na:na] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[na:na] at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na] at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[na:na] at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:110) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44) ~[junit-jupiter-engine-5.10.3.jar:5.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.3.jar:1.10.3] at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[na:na] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.3.jar:1.10.3] at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[na:na] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) ~[junit-platform-engine-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63) ~[junit-platform-launcher-1.10.3.jar:1.10.3] at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57) ~[junit5-rt.jar:na] at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) ~[junit-rt.jar:na] at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) ~[idea_rt.jar:na] at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) ~[junit-rt.jar:na] at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232) ~[junit-rt.jar:na] at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55) ~[junit-rt.jar:na] org.springframework.ai.retry.NonTransientAiException: 400 - {"error":{"message":"invalid image input","type":"invalid_request_error","param":null,"code":null}} at org.springframework.ai.retry.RetryUtils$2.handleError(RetryUtils.java:77) at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) at org.springframework.web.client.StatusHandler.lambda$fromErrorHandler$1(StatusHandler.java:71) at org.springframework.web.client.StatusHandler.handle(StatusHandler.java:146) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.applyStatusHandlers(DefaultRestClient.java:696) at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:200) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:683) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntityInternal(DefaultRestClient.java:653) at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntity(DefaultRestClient.java:642) at org.springframework.ai.openai.api.OpenAiApi.chatCompletionEntity(OpenAiApi.java:1022) at org.springframework.ai.openai.OpenAiChatModel.lambda$call$1(OpenAiChatModel.java:226) at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:344) at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:217) at org.springframework.ai.openai.OpenAiChatModel.lambda$call$3(OpenAiChatModel.java:226) at io.micrometer.observation.Observation.observe(Observation.java:565) at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:223) at org.springframework.ai.openai.chat.proxy.OllamaWithOpenAiChatModelIT.multiModalityImageUrl(OllamaWithOpenAiChatModelIT.java:345) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

tzolov · 2024-09-23T21:57:30Z

Rebased, squashed and merged at 5017749

JogoShugh · 2024-10-08T16:15:13Z

This is awesome. Thank you!

A few months ago when I was just getting started calling OpenAI, I was auto-generating JSON schema calling Jackson's JsonSchemaGenerator by hand. I just read over all this stuff in Spring AI this morning and after finding this issue, it's doing it all for me perfectly.

The ability to get the arguments in JSON is critical for me as I want to be able to have a "preview" kind of flow where users can confirm a spoken command's AI-assisted translation-to-json or hand edit it before I call my own "confirm" step with the payload.

If anyone needs it for Kotlin, this is how I just got the build and tested:

Update build.gradle.kts

Set to the build found in the repo:

//extra["springAiVersion"] = "1.0.0-M2"
extra["springAiVersion"] = "1.0.0-20241008.115115-715"

Register a function

 data class  PrepareBed(
      val bedId: UUID,
      val name: String,
      val dimensions: Dimensions,
      val cellBlockSize: Int = 1
 )

@Configuration
class BedFunctionsConfig(val objectMapper: ObjectMapper) {

    @Bean
    fun prepareBedCallback(repository: BedRepository): FunctionCallback =
        FunctionCallbackWrapper.builder { command: PrepareBed ->
                val bed = BedAggregate.of(command.bedId, command.name, command.dimensions, command.cellBlockSize)
                repository.addBed(bed)
                val resource = BedResourceWithCurrentState.from(bed)
                // resource.includeSchemas() // <-- This was my hand-rolled Jackson schema functionality
                resource
            }
            .withName("prepareBed")
            .withDescription("Prepare a garden bed")
            .withObjectMapper(objectMapper)
            .withInputType(PrepareBed::class.java)
            .build()
}

Implement a controller that calls Spring AI

@RestController
class BedCommandHandler(
    chatClientBuilder: ChatClient.Builder,
    private val mapper: ObjectMapper
) {
    private val client : ChatClient =
        chatClientBuilder
            .defaultAdvisors(
                SimpleLoggerAdvisor()
            )
            .build()

    @PostMapping("/api/beds/{bedId}/action")
    suspend fun action(@PathVariable bedId: UUID,
                     @RequestParam("prompt") prompt: String) : Flux<String> {
        val systemMessage = SystemMessage(getSystemPrompt(bedId))
        val userMessage = UserMessage(prompt)

        val response = client.prompt()
            .options(
                FunctionCallingOptionsBuilder().withProxyToolCalls(true).build()
            )
            .functions("prepareBed")
            .messages(systemMessage, userMessage)
            .call()
            .chatResponse()

        val args = response.result.output.toolCalls[0].arguments

        val tree = mapper.readTree(args)
        val pretty = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(tree)
        return Flux.just("Does this look correct to you?\n\n$pretty")
    }
}

Observe the serialized payload in `toolCalls`

I issue a command to the api with ./function.sh "Prepare Venus bed 4 by 8, 1 foot high"

Then, I can see it properly serializing the generated payload into toolCalls:

As stated above, this will be absolutely outstanding for me because I want to be able to take these results as "previews" and present them to the user for Confirmation that the AI-interpreted translation of their spoken or written intention is accurate before calling the actual function to invoke the behavior.

See the pretty printed API response preview (in my use case)

Of course, I'll format this for UI when I get there, but for now:

./function.sh "Prepare Venus bed 4 by 8, 1 foot high"

* Connected to localhost (::1) port 8080
> POST /api/beds/2fbda883-d49d-4067-8e16-2b04cc523111/action HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 200 
< Content-Type: text/plain
< Transfer-Encoding: chunked
< 

Does this look correct to you?

{
  "bedId" : "2fbda883-d49d-4067-8e16-2b04cc523111",
  "name" : "Venus",
  "dimensions" : {
    "columns" : 4,
    "rows" : 8,
    "height" : 1
  }
* Connection #0 to host localhost left intact
}

Thanks again, this is excellent stuff!

tzolov added 3 commits September 23, 2024 21:00

Remove OpenAiAdapter

f953aaf

Improve the client side ToolCallHelper

f28eaa8

Add ITs for proxy tool calls

tzolov force-pushed the gh-1367-function-calling-proxyied branch from c4d4572 to f28eaa8 Compare September 23, 2024 19:00

tzolov marked this pull request as ready for review September 23, 2024 19:01

markpollack reviewed Sep 23, 2024

View reviewed changes

tzolov added this to the 1.0.0-M3 milestone Sep 23, 2024

tzolov closed this Sep 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add proxy tool calls option to chat models #1398

Add proxy tool calls option to chat models #1398

tzolov commented Sep 23, 2024 •

edited

Loading

markpollack Sep 23, 2024 •

edited

Loading

markpollack Sep 23, 2024

tzolov commented Sep 23, 2024

JogoShugh commented Oct 8, 2024 •

edited

Loading

Add proxy tool calls option to chat models #1398

Add proxy tool calls option to chat models #1398

Conversation

tzolov commented Sep 23, 2024 • edited Loading

markpollack Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

markpollack Sep 23, 2024

Choose a reason for hiding this comment

tzolov commented Sep 23, 2024

JogoShugh commented Oct 8, 2024 • edited Loading

Update build.gradle.kts

Register a function

Implement a controller that calls Spring AI

Observe the serialized payload in toolCalls

See the pretty printed API response preview (in my use case)

tzolov commented Sep 23, 2024 •

edited

Loading

markpollack Sep 23, 2024 •

edited

Loading

JogoShugh commented Oct 8, 2024 •

edited

Loading

Observe the serialized payload in `toolCalls`